Method and apparatus for measuring motion of a subject using a series of partial images from an imaging system

ABSTRACT

A line scan imager is used to determine the motion of a subject. Each line of image data from the line scan imager is compared with a reference image. The location of a matching line in the reference image reveals the displacement of the subject. The current subject displacement can be determined based on each line of image data. The resulting displacement information can be used to correctly place other optical beams on the subject. The method can be applied to tracking the human eye to facilitate measurement, imaging, or treatment with a beam of optical radiation.

PRIORITY

This application is a Continuation of U.S. patent application Ser. No.12/861,672 (now U.S. Pat. No. 8,050,504), filed Aug. 23, 2010, which isa Divisional of U.S. patent application Ser. No. 11/389,351 (U.S. Pat.No. 7,805,009), filed Mar. 24, 2006, which claims the benefit of thefiling date under 35 U.S.C. §119(e) of Provisional U.S. patentapplication Ser. No. 60/668,785, filed on Apr. 6, 2005, which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to systems for tracking the position of anobject. In particular, the invention is a system for determining motionof the eye of a patient, using partial images from a scanningophthalmoscope. The intended field of application of the invention is touse the tracking information to correct the position of opticaltreatments or optical measurements made on the patient's eye.

BACKGROUND OF THE INVENTION

Directed beams of light are used for both measurement and treatment ofpatients, and the patient presents a moving subject. In the field ofophthalmology, laser photocoagulation is an example of a treatmentmethod, and optical coherence tomography an example of a measurementmethod, both of which are typically performed with the patient awake,and both of which require precise placement of the light beam on aportion of the eye.

A typical patient can comfortably hold his eye open for a few seconds.The eye moves considerably in one second, mainly through quickadjustments in fixation (small saccades) resulting in apparent motionsof the retina on the order of one hundred microns. These motions causenoticeable errors in application of directed beams such asphotocoagulation and optical coherence tomography (OCT). Tracking themotion of the eye to correct the placement of the beam has proven usefulin photocoagulation [Naess, E., et al. (2002)] and in OCT [U.S. Pat. No.6,736,508; Hammer, D. X., et al. (2005)].

Typically, a pair of rotating minors serves as a two-dimensional scannerto move the beam of light in two dimensions, x and y, across thesubject. If motion of the subject is tracked, the positioning commandsto the scanner can be adjusted so that the scan beam reaches at thedesired positions on the subject.

Information on the motion of the subject must be provided with lowlatency so that the scanning beam is correctly positioned for eachA-scan in OCT, or for each laser shot in photocoagulation. In a systemthat corrects the scan beam position, the latency is the time betweeneye motion and correction of the position of the scan beam.

Tracking methods that use two-dimensional image frames [U.S. Pat. Nos.4,856,891; 5,729,008; 5,975,697; and U.S. Patent Application PublicationNo. 2005/002458] have the advantage that the two-dimensional image canalso be used for a real-time display to orient the operator during themeasurement or treatment procedure. These methods typically incurlatency approximately equal to the time between frames, which istypically 1/30 of one second. During one frame, the eye can movesignificantly [Hammer, D. X., et al. (2002)]. Faster frame rates arepossible, but incur extra cost.

Tracking methods that use a dithered tracking beam are fast enough tofollow the motion of a human eye [Hammer, D. X. et al. (2002); U.S. Pat.Nos. 5,943,115, 5,767,941]. Dithered-beam methods with update rates of2-10 kHz have been successful in tracking the human eye. The ditheredtracking beam requires a separate optical scanning system, in additionto the system used to scan the treatment or measurement beam.

A line-scan ophthalmoscope (LSO) produces an image of the eye one lineat a time [U.S. Pat. Nos. 4,732,466; 4,768,874; and 6,758,564]. In anLSO using an electronic camera, each line can be acquired and madeavailable to digital processing within less than one millisecond. Thepart of the eye image contained in each line can be compared to the samearea in previous eye images in order to determine the eye motion.Individual lines from an electronic LSO are available at approximately10 kHz.

Previously disclosed tracking methods typically use a landmark, such asthe optic disk. The landmark is identified first, and its location ismonitored as the measurement, or treatment, scan proceeds. However, goodlandmarks are not always found in diseased tissue.

We see a need for a system to track motion of the eye, or other humantissue, with low latency during an optical treatment or opticalmeasurement procedure, where the tracking system shares apparatus with asystem providing a real-time display to the operator, and using a methodthat is independent of any particular landmark in the tissue.

SUMMARY OF THE INVENTION

The principal object of this invention is to track the position ofmoving human tissue, using information from a line-scan imaging system.The line-scan imaging system can provide a real-time view of thesubject, in addition to tracking information.

The method of estimating motion of the subject includes the steps: (a)collect a current line of the image of the subject from the linedetector in the line-scan imager, (b) compare that current line withpreviously-recorded lines collected from nearby areas of the subject,the comparison allowing for shifts in two dimensions, and optionally forrotation, of the current line (c) determine the shift in two dimensions,and optionally the rotation, which best aligns the current line withpreviously-recorded lines.

The apparatus includes: (a) a line-scan imager to collect lightreflected from the subject and form an image of the subject; (b) dataprocessing equipment to accept lines from the line-scan imager, analyzethe lines to derive estimates of the current position of the subject,and output these estimates; and (c) an optical scanning system to placean optical beam on a subject, and optionally to apply the signal fromthe data processing equipment to correct the placement of the opticalbeam.

The optical beam can be either a treatment beam, such as in lasersurgery, or a measuring beam, such as in optical coherence tomography.If the optical beam is a measurement beam, then a variation of thismethod can be used in post-processing to correct collected optical datafor motion of the subject. That is, the method can be used for dataregistration after measurement, as well as for subject tracking duringthe measurement. The collected optical measurements and lines from theline-scan imager are stored with knowledge of their relative timepositions, such as time stamps, so that each optical measurement can beassociated with a line from the line-scan imager this line serving asthe “current” line for that optical measurement. In this variation thereference frame can be built from line-scan image data taken before,during or after the scan of the measurement beam.

A specific application of this method is in an ophthalmic instrumentthat combines an OCT scanner and LSO, including electronics and softwareto cross-correlate each line acquired by the line-scan camera in the LSOwith a portion of the previous frame from the LSO, so as to determinethe current position of the retina relative to its position in theprevious frame. This apparatus provides an estimate of the apparentposition of the retina upon acquisition of each line in from the LSO.These position estimates are available with only a short delay after theeye motion occurred, so they can be used to correct the OCT scanpositions so as to measure desired region of the retina in the presenceof eye movements. This correction can be implemented by various methods,such as correcting the signals driving the OCT beam scanner, or bydeflecting the OCT beam with correcting optics.

The embodiment described below applies this invention to a line-scanophthalmoscope and OCT measurement system, but it will be evident tothose skilled in the art that this invention can be applied to line-scanimaging of other moving subjects, and for correcting the placement ofother types of optical measurement or optical treatment beams. Thetracking information derived from the line-scan imager can be used tocorrect the line-scan image itself, for example by forming an averagedimage in which the tracking information allows one to correctly placelines from several passes of the line scanner into the averaged image.The tracking information can also be used to adjust the scan range ofthe line-scan imager so that the region of interest remains stationaryin the view of the line-scan imager.

The line-scan imager may be any imaging system that builds a fulltwo-dimensional image in parts, and that can output partial images fordigital processing before the entire image is scanned. One example is aspot-scanning confocal imager, in which the optical intensity reflectedfrom the subject is measured one pixel at a time, with the scantypically proceeding through rows in a raster scan [U.S. Pat. Nos.4,135,791; and 4,213,678]. A partial image, serving as the ‘line’ in thedescription of this invention, could be the set of reflected intensitiesin one or a few rows of the raster scan, or the set of intensities froma partial row. In this example of a spot-scanning imager, individualpixels of the partial image are recorded at different times, but if themeasurements are closely spaced in time compared with the rate ofchanges in subject position, then the measurements are substantiallysimultaneous for purposes of the method disclosed here. Another exampleof an imaging system that builds its image in parts is an OCT scanner.An OCT scanner localizes reflections in three dimensions, and is capableof building a three-dimensional image. Each A-scan from an OCT scanner,an A-scan containing reflectivity as a function of distance along thebeam, can serve as the ‘line’ in the method described here, with aprevious two-dimensional or three-dimensional OCT image serving as thereference frame.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a combination imaging and optical measurement systemapplied to the human eye.

FIG. 2 shows a fundus image from a human eye, and a representation of apartial fundus image from a line-scan ophthalmoscope

FIG. 3 shows the cross-correlation function between the partial imageand the full image.

FIG. 4 plots estimates of eye position as a function of time.

DETAILED DESCRIPTION

FIG. 1 is a schematic illustration of an OCT measurement systemincorporating eye tracking based on a line-scan ophthalmoscope (LSO).Light from the LSO light source 101 is routed by cylindrical lens 102and beamsplitter 103 to scanning mirror 104. The cylindrical lens 102and the scan lens 105 produce a line of illumination at the retinalimage plane 106, and the ocular lens 107 and optics of the human eye 100re-image this line of illumination onto the retina 110. The line ofillumination is swept across the retina as the scanning mirror 104rotates. Reflected light from the retina approximately reverses the pathof the LSO illumination light; the reflected light is de-scanned by theLSO scan minor 104 so that the illuminated portion of the retina iscontinuously imaged by imaging lens 108 onto the LSO line camera 109.The LSO line camera converts the reflected LSO light into a data stream140 representing single-line partial images, which are processed to formboth eye tracking in formation and a real-time display of the retina.

The OCT system 120 incorporates the light source, light detector ordetectors, and processor required to determine the depth profile ofbackscattered light from the OCT beam 121. One type of OCT system isdescribed in the reference by Huang et al. OCT scanner 122 sweeps theangle of the OCT beam in two dimensions, under the control of scancontroller 154. Scan lens 123 brings the OCT beam into focus on theretinal image 106. Beamsplitter 124 combines the OCT and LSO beam pathsso that both paths can more easily be directed through the pupil of thehuman eye 100. (Combining the beam paths is not required in directimaging applications, where the object itself lies in the location ofretinal image 106.) If the OCT and LSO use different wavelengths oflight, beamsplitter 124 can be implemented as a dichroic mirror. The OCTbeam is re-focused onto the retina through ocular lens 107 and theoptics of the human eye 100. Some light scattered from the retinafollows the reverse path of the OCT beam and returns to the OCT system120, which determines the amount of scattered light as a function ofdepth along the OCT beam.

As noted above, the eye 100 may move with respect to the measurementsystem causing the correspondence between the position of scanner 122and positions on retina 110. In some optical measurement or opticaltreatment systems, such as those using handheld scanners, motion of thescanning optics can contribute to the relative motion between scannercoordinates and sample coordinates.

The stream of partial images 140 from the LSO camera are provided to aprocessing routine described below and represented by block 150 inFIG. 1. One output of the processing routine is a series of completeframes 141 providing the operator with a real-time view of the subject,via display 151. Another output of the processing routine is series 142of estimates of the current displacements of the eye. These estimates ofcurrent displacement can be combined with the set 143 of locations onthe subject at which OCT measurements are desired, to form a set ofcorrected scan coordinates 144 provided to the scan controller 154 sothat the scanner 122 directs the OCT beam to the desired location on theeye.

An LSO operating at 20 frames per second, with 512 lines per frame,provides 10,000 lines per second. We use a digital line-scan camera, sowith appropriate hardware each of these lines is available forprocessing within 100 microseconds after the instant the light wasreflected from the eye.

Each line from the LSO is compared with a reference frame, at a range ofshifts in x and y. FIG. 2 shows a two-dimensional image 200 of a humanretina built from an LSO. A single line from the LSO produces a verticalline such as 201. (Line 201 is represented in FIG. 2 by a strip ofseveral lines for better visibility.) In the method of this inventionline 201 is conceptually shifted across reference frame 200 until a goodmatch is found. If the eye has not moved, one can predict that line 201will match that portion of reference frame 200 that has the same scannercoordinates as line 201, the same position of the LSO scanner and thesame shift along the line-scan camera. If the eye has moved, relative tothe coordinate system of the scanner, line 201 and the matching portionof reference frame 200 will have different scanner coordinates.

The comparison between the line 201 from the LSO and the reference frame200 can be done using a cross-correlation, forming a cross-correlationfunction C(d) where the vector d=(Δx, Δy) represents the shift. Thecross-correlation function is computed from the image intensities in thecurrent line L(r) and in a group of previously-acquired lines forming atwo-dimensional reference frame A(r), in which the vector r=(x,y)denotes position on the retina. It is advantageous to use the normalizedcross-correlation, which takes the value of 1 when the two sub-imagesL(r) and A(r) match exactly,

${{C(d)} = {\sum\limits_{r}{{A\left( {r - d} \right)} \cdot {{L(r)}/\sqrt{\sum\limits_{r}{{A^{2}(r)} \cdot {\sum\limits_{r}{L^{2}(r)}}}}}}}},$

in which the sum is taken over the locations r in the current line. Anexample of a cross correlation function between a line and referenceframe acquired from the retina of a human eye are plotted in FIG. 3.FIG. 3 is a contour plot of value of C(d) as a function of thecomponents Δx, Δy of the shift d. The normalized cross-correlationreaches a peak of 0.8 at a shift of Δx=−100 μm, Δy=−130 μm, comparedwith lower values at other shifts. This match indicates that the retinahas moved 100 μm to the left and 130 μm down between acquisition of thereference frame and acquisition of the current line from the LSO.

Efficient methods of computing cross-correlations, including computationusing the fast Fourier transform, are well known in the art and theseefficient methods are preferably used with this invention. Anotheradvantageous technique is to compute the cross-correlation ofintensities after spatially high-pass filtering, so as to retain detailassociated with the tissue but attenuate large-scale variation such asnon-uniform illumination. In addition, low-pass filtering theintensities can be useful for reducing noise in the computedcross-correlation function.

The cross-correlation will often have a peak as a function of d, thelocation of that peak indicating the shift d that best aligns thecurrent line with the reference frame. This peak location is one choicefor our estimate D of subject motion since acquisition of the referenceframe. An alternative estimate is the centroid of the cross-correlationfunction, D=ΣdC(d)/ΣC(d) where the sum is taken over all consideredshifts d. Such calculations can be weighted by our a-priori expectationG(d) that the subject has shifted by d since acquisition of thereference frame: D=ΣdG(d)C(d)/ΣG(d)C(d).

The shift d represents the subject motion between acquisition of theline L(r) and acquisition of the reference frame A(r). An importantaspect of this invention, however, is that the cross-correlations can beevaluated upon acquisition of each line. Therefore the time betweenestimates D is only the time between acquisition of successive lines.Successive values of D are expected to differ by only the small motionpossible in the short time between acquisition of successive lines. Thecorrection signal provided by this tracker (for application either tocorrect an optical scan or to register a previously-acquired data set)can be a series of very small corrections, corresponding to the motionof the subject between successive line acquisitions.

Previous tracking information can be used in the a-priori expectationG(d). For example G(d) may be a function peaked at the expected shift,based on recent motion of the subject, with a width consistent withrecent uncertainty in the system and physically reasonable velocity andacceleration of the subject. It is not necessary to compute thecross-correlation at shifts d outside the range of a-prioriexpectations, allowing savings of computational time.

Alternative to the cross-correlation C(d), other measures of similaritybetween the sub-images can be used. Two commonly-used measures are thesum of squared differences and sum of absolute differences, thesequantities being minimized for well-matched sub-images.

Human eyes rotate slightly about the optical axis. The shift d canoptionally be generalized to include rotations by rotating the previousframe, interpolating data points where necessary, before thecross-correlation. In this case the offset of the vector r indicatedabove by the vector subtraction (r−d) must be generalized to includerotation of r.

Alternatively to rotating the previous frame, cross-correlation under arotation angle can be estimated by cross-correlating separate portionsof the current line with the previous frame, considering only shifts dwithout rotation, to derive individual cross-correlation functionsC_(i)(d). The overall cross correlation function can be evaluated at anyrotation θ by forming the sum C(d)=ΣC_(i)(d+θe_(i)) where the vectorsθe_(i) are the displacements of the centers of each portion of thecurrent line caused by rotation through and angle θ. (Considering theadded shifts θe_(i) only at the centers of the portions of the currentline is an approximation, because the image details that most influencethe cross-correlations may not be centered in these portions of thecurrent line.)

The range of shifts d can optionally be limited, given knowledge of thecharacteristics of subject motion, specifically of human eye fixation.For example, it is practical to scan the retina of the human eye at 30frames per second, with 512 lines in each frame (requiring a 15 kHz linerate from the LSO camera). Given a relatively fast shift in gaze,300°/sec, the eye could rotate 10° between frames, shifting the apparentposition of the retina by approximately 3 mm. Between successive lines,however, the apparent position of the retina would move only 6 μm sincethe last position estimate. Given the possibility of short segments ofbad position estimates, as discussed below, it is practical to search arange of shifts d spanning 100 μm horizontally and vertically. Rotationof the image of the retina can be caused by so-called torsional rotationof the eye. The torsion angles are a fraction of the rotation angles ofthe eye, and between successive lines the rotation of the retinal imagewill be less than 2 millidegrees. If one chooses to correct for thisrotation, it is sufficient to search a 50-millidegree range of angles θ.

Additionally, one can cross-correlate each line with local lines withinthe same frame. Cross-correlation of neighboring lines gives mainlyinformation on motion along the length of these lines. Accelerationperpendicular to the length of the line is revealed in the relativecorrelation between a given line and its immediate neighbors to eachside.

Pattern noise in typical line-scan cameras tends to bias thecross-correlation toward certain shifts. To reduce this bias one canoptionally correct for pattern noise using standard techniques includinglook-up tables.

There is uncertainty in the estimate of motion. When the LSO scansrelatively featureless sections of the eye, the image data returnedoften lacks detail, and there is no shift where the match to thereference frame is clearly superior; that is there is sometimes no peakin C(d). One can estimate the uncertainty from the shape of the peak inthe cross-correlation function C(d). By way of example, thecross-correlation value C(d), optionally multiplied by the a-priorilikelihood G(d), can be used as a measure of the likelihood that theactual eye shift was d. The eye shift could be estimated byD=ΣdG(d)C(d)/ΣG(d)C(d), assuming that the data has been preprocessedsuch that C(d) approaches zero for un-related images. An example methodof preprocessing is to high-pass filter the images before convolutionthen optionally to set to zero values of C(d) below a certain threshold.The corresponding uncertainty is [Σ(d−D)²G(d)C(d)/ΣG(d)C(d)]̂(0.5).

As an additional example, the magnitude of the peak in C(d) is a measureof the confidence in the estimate D; the inverse of this magnitude canserve as an uncertainty in D. As a further example, the area in d overwhich C(d) exceeds a chosen threshold indicates the range of likelyshifts D; the square-root of this area can serve as an uncertainty in D.

The position information and uncertainty estimates can be monitored toindicate to the operator the current quality of the tracking data, andthe current state of the subject. In an ophthalmic device, the amount ofeye wander can be quantitatively indicated to the operator, indicatingwhen the patient may be beginning to tire.

The position estimates D are obtained upon acquisition of each line fromthe line-scan imager. These line acquisitions may come more quickly thanthe subject moves. Groups of position estimates D can be averaged,preferably weighted according to their uncertainties, and furtheroptionally weighted according to their age, to form estimates withreduced noise. (Alternatively, estimates D with particularly largeuncertainty can be simply discounted for purposes of tracking.)

This tracking method makes use of a reference frame. Any given framecaptured by a line-scan imager can be distorted by motion occurringduring acquisition of that frame, so the possible distortions within thereference frame must be considered. One approach is to build a referenceframe from an initial set of N acquired frames. Within the data setcomprising the first N frames, lines from frame 2 onward are compared tothe first frames to form a stream of estimates of displacement D(t) inthe manner described above. This stream of estimates suffers from errorsif there was eye motion during the first frame, but such motion can bediscovered by analysis of D(t). For example, the first time derivativeof D(t) yields a velocity V(t) that will have peaks corresponding toeach saccade of the eye. The problem of motion during the initial framemanifests itself as peaks in V(t) that repeat every time the LSO scannerre-traces a particular portion of the eye. If one plots the estimatedvelocity V(t) as a function of position on the eye, then the resultingN−1 plots will show common features due to motion during the firstframe. The median over these N−1 plots is a good estimate of thenegative of the eye velocity during that first frame. From the velocityestimate given by this median, on can integrate to form positionestimates for the first frame, and correct the data stream D(t) for theeffects of motion during the first frame. Now having position-correctedimage data from all N frames, one can combine the N frames of image datato form a reference frame free from motion artifacts. Other methods ofbuilding motion-artifact-free images are known in the field of medicalimaging (for example, U.S. Pat. No. 5,575,286). The construction of areference frame from data acquired on a moving eye will move image dataaround, so that portions of the reference frame are associated withscanner coordinates different from the scanner coordinates at whichthose portions were originally acquired.

There are periods of unreliable data, both serious (blinks) and mild(scanning a relatively featureless region). There may also be periodswithout data. For example, in a line-scan ophthalmoscope scanningrepeatedly in one direction, the period required for the scanner toreturn to the starting position is a period of time during which noposition information is available. It is advantageous to minimize theduration of these periods. For example, in a line-scan imager it isadvantageous to operate with bi-directional scanning.

One can provide a continuous stream of position estimates throughperiods of unreliable or missing data, by using the uncertaintyestimates described above to weight the contributions of individualestimated shifts D(t) in determining the estimated eye position X(t). Asan example, one particular method of combining the stream of estimatedshifts D(t) is to perform a weighted fit to a function f(t). The fittingfunction could be for example a linear or quadratic polynomial. Aftereach line scan, the value of the fitting function at the end of the datastream provides an improved estimate of the shift. During periods ofunreliable data, earlier data will more strongly influence the fit, andthe values of the fitting function effectively extrapolate from theseearlier measurements. When the estimated shifts become reliable again,these low-uncertainty estimates strongly influence the fit, and thevalues of the fitting function return to follow the stream D(t). It isuseful to have the uncertainties in the stream of estimated shifts D(t)grow as the age of the data increases, so that old estimates smoothlylose their influence in the fit.

Knowledge of possible motion of the subject, such as physical limits onaccelerations and velocities, allows more robust fitting by reducing theweight of estimated shifts that are inconsistent with possible motion ofthe subject.

FIG. 4 shows plots of the measured and fit positions of a human eyeretina based on the data stream from an LSO. (More specifically, we plotof the apparent position of the retina, as seen through the human eyeoptics.) Each of the points marked with ‘+’ signs in FIGS. 4 a and 4 bare estimates D(t_(n)) of the eye motion, where the integer n counts thesuccessive lines returned by the LSO. FIG. 4 a shows horizontal (x)components of motion and FIG. 4 b shows vertical (y) components. (A fewof these estimates lie above or below the range of the plots.) For thisdata, the fitting functions f_(n)(t) were straight-line segments fit tothe most recent fifty estimates D(t_(n)). We used a weighted fit, withthe weights assigned to each estimate D(t_(n)) based on the value of thecross-correlation C(D(t_(n))), based on the age of the data point, andalso based on our estimate of the a-priori likelihood of the eye beingin this position. The a-priori likelihood is determined from how closethe current estimate of position comes to extrapolation of the previousfit. Specifically, the relative weight assigned to each estimateD(t_(n)) was C(D(t_(n)))*exp[t_(n)/τ]*exp[ID(t_(n))−f_(n-1)(t_(n))|/a]where the best parameters a≈15 μm and τ≈2.5 msec were found throughexperimentation. With each new estimate D a new fit is performed. Thesolid curves in FIGS. 4 c and 4 d are plots of the resulting sequencef_(n)(t_(n)): the values of the updated fitting functions f_(n), eachevaluated at the times t_(n) corresponding to the latest line returnedfrom the LSO.

The scan range of a line-scan imager is generally easily adjustable. Itmay be advantageous in the implementation of this method to narrow thescan range to cover regions of the subject containing detail useful fortracking. The narrow range can be scanned during periods when trackinginformation is critical, and widened at other times. The narrow rangecan be shifted to remain on the regions of the subject with relativelymore detail, using the data from the tracking system itself. Using thetracking data to adjust the scan forms a control loop. The low latencyof this line-based tracking method is advantageous in the implementationof such a control loop, as is well known in the art of feedback control.

This tracking method allows adaptations to save computational time.

(1) A shift estimate can be computed for only one or more selected linesin each newly-acquired frame. That is, if a line L(r) is expected toproduce a noisy maps C(d) one may want to skip computation of C(d). Onecan process just those lines containing relatively detailed structure,these lines giving the most accurate position estimates. In this way onespends his computational resources on those lines that yield the mostinformation. The amount of detail can be estimated with a small fractionof the effort required to compute C(d), for example by computing theautocorrelation of a single line. The locations of useful detail in theobject remain the same from scan to scan, so an estimate of the value ofeach line is available from previous lines covering the same area.

(2) The cross-correlations may be done by fast Fourier transform (FT)along the direction of the line-scan camera. The forward FT of previouslines can be saved. For each line, one can perform the forward FT, andpointwise multiply with saved FTs within the range of plausible shiftsto compute the FT of the cross-correlation functions. Theroot-mean-square (RMS) of these pointwise products gives the RMS of C(d)along the corresponding line. The reverse FT is performed only for a fewproducts with the largest RMS.

(3) The operation count for the fast FT can be significantly reducedusing a-priori knowledge if the range of shifts d over which the peak inthe cross-correlation function C(d) may be found. This means thatshorter arrays can be used in numerical computation of C(d). Thepreferred implementation smoothes and then sub-samples the stored FTs oflines from the line-scan camera; these sub-sampled arrays have typically⅛ the number of elements as the full array. The operations that arerepeated several times per line from the line-scan camera, the pointwisemultiply and reverse FT, are thus performed on significantly shorterarrays, providing a significant improvement in computational speed.

We estimate the computational resources required. The cross correlationneed only be computed over a limited range of shifts d. Taking forexample the case of shifts only, without rotation, we estimate thelargest believable eye shift between lines, in units of pixels.

Dead time between sweeps should be avoided; bi-directional LSO sweepingis preferred. The distorted information at the ends of the sweep, due tothe gradual change in direction of the scanner, can be used with thetracking system, taking care to account for the different delta-xbetween lines. By way of example we consider a bi-directionaltriangle-wave scan with 500 acquisitions of the line-scan cameracovering 10 mm of tissue. If there is 1 millisecond during which thescanner covers tissue not included in the reference frame, then eyerotation of 350 degrees per second (100 mm/sec apparent motion of theretina) could move retinal tissue by 0.1 mm, which corresponds to 5acquired lines.

We can now estimate the number of multiplications required to build thecross-correlation map C(d) for each line acquired by the LSO. One canuse FT-based correlation calculations in 1 dimension along thefull-length of each line. Each line in the reference frame has alreadyhad a FT calculated; if these previous results are stored, we need onlydo the FT on the current line. One then multiplies the current line byeach stored line from the reference frame within the range over whicheye motion is possible, and performs an inverse FT on the product withthe largest rms value. If one searches over 5 lines from the referenceframe, then the number of multiplications required for the steps listedin this paragraph is approximately 2N log₂N+N*2*5+2N log₂N, where Ndenotes the number of pixels in a line. For 512 pixels in a line, thisis 25,000 multiplies. If one uses a TMS320-family DSP running at 1 GHzto execute 3600 Mmac/sec, building C(d) for each line from the LSOrequires only 7 microseconds.

Fitting the data stream D(t) can be done with a small operation count.Linear least-squares fitting uses sums of the form Σ(f(t_(i))D_(i)/σ_(i)²) where f(t) is one of the components in the fitting function, D_(i) isa component of the estimated eye position, and σ_(i) is the uncertainty.If there are P free parameters in the fitting function, the fitparameters come from solution of a matrix equation involving a P×Pmatrix containing such sums; solution of this matrix equation requiresonly P³ multiplies. These sums themselves can be updated with P²additions for each line scanned. P is likely to be small, 2 or 3, sothis operation count is insignificant compared with the operation countfor the cross-correlations.

We have disclosed a method of using sections of an image, in the orderin which they are acquired, to derive multiple measurements of themotion of the subject in the time required to build a full image. Someimaging systems that can be used with this method are: a flying-spotconfocal microscope, a line-scan imager, and an OCT scanner. Onepotential subject of the method is the human eye. Some uses of themeasurements of subject motion are to correct the positions of anoptical beam, and to correct the interpretation of an optically-acquireddata collected concurrently with the position measurements. The imageacquired can also serve as a real-time display for the operator.

The optical beam being corrected can be a diagnostic beam, such as theprobe beam in an OCT system. The determination of the motion is made bycomparing a current sub-image with a previous sub-image. The currentsub-image can be a single line from the line-scan imager, or a number oflines from the line-scan imager. The comparison can be made bycross-correlation of intensities in the two sub-images to be compared,or by computing the sum of absolute differences between the intensitiesin the two sub-images to be compared, or by other means of comparison todiscover the shift of the subject between acquisition of the twosub-images. The estimate of the shift can be the centroid of thecross-correlation function, or the peak of the cross-correlationfunction, or another means to estimate the shift based on comparison ofthe two sub-images. The uncertainty in these estimates can be determinedusing the second moment of the cross-correlation function, or from thearea within the cross-correlation function under a certain threshold, orother means. The uncertainties can be used to derive, from a series ofshift estimates, a best current estimate of the subject position, forexample by least-squares fitting the shift estimates to a function oftime. A-priori information about the possible motion of the subject canbe used to decrease the weight in the fit given to outliers in theseries of shift estimates. The range of shifts that are compared islimited based on current knowledge of the subject position. Thesub-images to be used in the computations of correlation can be selectedbased on the amount of useful-detail in these sub-images; one method ofestimating the useful detail is to compute the autocorrelation functionand to estimate the uncertainty in a shift as we would for across-correlation function. The scan range of the imaging system can belimited to cover regions of the subject with useful detail. The scanrange can be selected to cover the area of the subject containing themost useful detail.

One apparatus in which this method can be applied is an ophthalmicsystem comprising an OCT scanner and an LSO. The LSO can provide areal-time view of the subject being scanned by OCT. The apparatus canadditionally provide electronics such as a digital signal processor(DSP) or microprocessor, and software, or other means to derive positionestimates from each line from the LSO. Alternatively to using softwareand a processor, the derivation of position estimates can be implementedin programmable logic devices. The apparatus can include a second set ofscan mirrors, additional electronic inputs to the beam scanner, or othermeans to correct the OCT scan position based on these positionestimates.

Although various embodiments that incorporate the teachings of thepresent invention have been shown and described in detail herein, thoseskilled in the art can readily devise many other varied embodiments thatstill incorporate these teachings.

The following references are hereby incorporated herein by reference.

U.S. PATENT DOCUMENTS

-   U.S. Pat. No. 4,135,791-   U.S. Pat. No. 4,213,678-   U.S. Pat. No. 4,732,466-   U.S. Pat. No. 4,768,873-   U.S. Pat. No. 4,856,891-   U.S. Pat. No. 5,575,286-   U.S. Pat. No. 5,644,642-   U.S. Pat. No. 5,729,008-   U.S. Pat. No. 5,767,941-   U.S. Pat. No. 5,943,115-   U.S. Pat. No. 5,975,697-   U.S. Pat. No. 6,283,954-   U.S. Pat. No. 6,325,512-   U.S. Pat. No. 6,758,564-   U.S. Pat. No. 6,726,325-   U.S. Pat. No. 6,736,508-   U.S. Pat. No. 6,769,769-   U.S. Published Application 20050024586-   U.S. Published Application 20050140984-   U.S. Published Application 20030199769    Co-pending U.S. application Ser. No. 11/331,567, filed Jan. 13,    2006, entitled “Method of Motion Correction in Optical Coherent    Tomography Imaging”

OTHER PUBLICATIONS

-   Huang, D. et al. Science 254 (5035): 1178-81 (1991).-   Hammer, D. X. et al. “Image Stabilization for Scanning Laser    Ophthalmoscopy” Optics Express 10 1549 (2002)-   Naess, E. et al. “Computer-assisted laser photocoagulation of the    retina—a hybrid tracking approach” Journal of Biomedical Optics    7(2): 179-189 (2002).-   Hammer, D. X. et al. “Active retinal tracker for clinical optical    coherence tomography systems” Journal of Biomedical Optics 10(2):    0240380-1 (2005).-   Rogers, J. A., et al. “Topography and volume measurements of the    optic nerve using en-face optical coherence tomography.” Optics    Express 9(10): 533-545 (2001).

1. (canceled)
 2. An apparatus for measuring the eye comprising: anoptical coherence tomography (OCT) system having a radiation beam whichis scanned over the eye to generate OCT measurement data; a confocaloptical imager for generating a two-dimensional image of the retinaindependent of the OCT system, said image being generated by scanning anillumination spot on the retina and collecting the reflected light,wherein the two dimensional image is defined by a collection of partialimages; and a processor for controlling the OCT and the scanning opticalimager, said processor for capturing image data from the imager andcomparing a partial image from the two-dimensional image to a referenceimage to determine if the eye has moved and using that information toeither correct the positioning of the radiation beam used by the OCT orto correct measurement data from the OCT system in post-processing. 3.An apparatus as recited in claim 2, further comprising a display fordisplaying the image generated by the confocal imager in real time tothe user.
 4. An apparatus as recited in claim 2, wherein the processorcompares the partial image to the reference image while thetwo-dimensional image is being generated.
 5. An apparatus as recited inclaim 2, wherein the processor determines if the eye has moved bycomparing the partial image with the reference image to find asubstantially matching image portion and determining a displacementbetween scan coordinates associated with the matched image portions,said displacement corresponding to the motion of the eye.
 6. A method ofmonitoring the movement of an eye of a patient in order to facilitatethe alignment of a beam of radiation with respect to the eye, said beambeing generated by a light source and directed to the eye via a scanner,said method comprising the steps of: generating a two dimensionalreference image of the eye; scanning imaging light across the eye andcollecting reflected image data to generate a second two dimensionalimage, wherein the two dimensional image is defined by a collection ofpartial images and wherein the imaging light is independent from thebeam of radiation; comparing a partial image from the two-dimensionalimage to the reference image to identify a substantially matching imageportion; determining a displacement between the scan coordinatesassociated with the matched image portions, said displacementcorresponding to the motion of the eye; and correcting the alignment ofthe beam of radiation with respect to the eye using the scanner based onthe determined displacement.
 7. A method as recited in claim 6, whereinthe beam of radiation is used to measure the eye.
 8. A method as recitedin claim 7, wherein the measurement beam is part of an optical coherencetomography system.
 9. A method as recited in claim 6, wherein the beamof radiation is used to treat the eye.
 10. A method as recited in claim6, wherein the second two dimensional image is generated by scanning aline of light across the eye.
 11. A method as recited in claim 6,wherein the second two dimensional image is generated by scanning a spotof light across the eye.
 12. A method as recited in claim 6, wherein thepartial image is a line scan.
 13. A method as recited in claim 6,wherein the partial image is less than a line scan.
 14. A method asrecited in claim 6, further comprising displaying the secondtwo-dimensional image to the user in real-time.
 15. A method as recitedin claim 6, wherein the comparison step is performed while the secondtwo-dimensional image is being generated.
 16. An apparatus for measuringthe eye comprising: an optical coherence tomography (OCT) system havinga radiation beam which is scanned over the eye to generate OCTmeasurement data; a line-scan ophthalmoscope (LSO) for generating animage of the retina by scanning a line of light across the retina andcollecting the reflected light; and a processor for controlling the OCTand LSO, said processor for capturing image data from the LSOcorresponding to a position of the line of light on the retina andcomparing the image data to a reference image to determine if the eyehas moved and using that information to correct measurement data fromthe OCT system in post-processing.
 17. An apparatus as recited in claim16, wherein the processor determines if the eye has moved by comparingthe image data with the reference image to find a substantially matchingimage portion and determining a displacement between scan coordinatesassociated with the matched image portions, said displacementcorresponding to the motion of the eye.
 18. An apparatus for measuringthe eye comprising: an optical coherence tomography (OCT) system havinga radiation beam which is scanned over the eye to generate OCTmeasurement data; an optical imager for generating a two-dimensionalimage of the retina independent of the OCT system, said image beinggenerated by illuminating the retina and collecting the reflected lightin parts so that the two-dimensional image can be defined by acollection of partial images; and a processor for controlling the OCTand the optical imager, said processor for capturing image data from theimager and comparing a partial image to a reference image to determineif the eye has moved and using that information to either correct thepositioning of the radiation beam used by the OCT or to correctmeasurement data from the OCT system in post-processing.
 19. Anapparatus as recited in claim 18, wherein the processor determines ifthe eye has moved by comparing the partial image with the referenceimage to find a substantially matching image portion and determining adisplacement between scan coordinates associated with the matched imageportions, said displacement corresponding to the motion of the eye. 20.An apparatus as recited in claim 19, wherein the processor repeatedlycompares various partial images with the reference image and combinesthe results to determine the displacement.
 21. An apparatus as recitedin claim 18, wherein the optical imager is a line scan imager.
 22. Anapparatus as recited in claim 18, wherein the optical imager is aspot-scanning confocal imager.
 23. An apparatus as recited in claim 18,further comprising a display that displays the two-dimensional image ofthe retina to the user in real time.